Remote apparatus operation and remote sensing are desirable in many applications and situations. For example, dangerous activities such as military operations, crime prevention, underground mining, exploration, and other activities benefit from remote apparatus operation and remote sensing. Similarly, situations and applications where rare expertise or skill are required can also benefit, such as where operation of a apparatus or analysis of a situation require a person not physically present in at the location of interest.
Prior art attempts to provide for remote apparatus operation and remote sensing have several significant drawbacks. One problem is that prior art systems often require more bandwidth than is readily available in many real world situations. In particular, typical prior art systems include a remote controlled apparatus having a camera and other sensors which are used to provide feedback for the remote operator. Video, audio, and other data (such as position, orientation, and state of the apparatus) are typically sent to the operator via a communications link. However, relatively large bandwidth is required to carry video, audio, and other data from the apparatus to the operator and to carry instructions from the operator to the apparatus. As a result, there are often problems with the necessary bandwidth not being available, or with interruptions to the transmissions. In such situations, the feedback from the apparatus can become inconsistent. For example, the video and audio can be become choppy, making it difficult for the operator to perform the desired tasks.
In addition, in the case of wireless communications links, wireless transmissions often pass through one or more repeaters as well as other equipment necessary for processing the signals, such as for compressing and decompressing the data. As a result, prior art systems include a noticeable latency between the input from the operator and the feedback signals from the apparatus. This latency creates problems in at least two ways. For example, when the operator receives feedback from a moving apparatus, that feedback is not current (due to the latency in the system) and a moving apparatus is actually in a different position that what is being displayed to the operator. In addition, the operator's instructions to the apparatus are not received by the apparatus until some time after the operators sends those instructions. As a result, the apparatus is receiving instructions well after the operator would like to provide those instructions to the apparatus. In addition, the operator is seeing the results of those instructions being executed well after the operator expects to see the instructions executed.
The bandwidth required by the prior art systems and the latency inherent in those prior art systems cause a number of problems. One problem caused by the latency in the prior art systems is that it is difficult to operate the remote apparatus effectively. In particular, an operator quickly becomes aware of the latency in the system, and that the operator's inputs are not being acted on by the apparatus until a noticeable time has passed. In order to operate an apparatus accurately, operators tend to drive very cautiously, stopping before an input is required and waiting for the situation to become static before providing additional inputs. For example, before negotiating a turn or passage through a tight space, an operator will typically stop the apparatus and wait for the situation to become static. At that point, operators typically provide a series of small inputs, stopping after each input to wait for the situation to again become static, until finally achieving the desired orientation of the apparatus. Once the apparatus is properly oriented, the operator will typically proceed slowly through the maneuver, repeating the above steps if it is necessary to again change the orientation of the apparatus.
Another problem with latency in the prior art systems is that some operators tend to become disorientated and nauseated by the effects of operating a apparatus in a system exhibiting significant latency.
One solution proposed by the prior art is to connect an optical fiber between the apparatus and the remote operator. A direct fiber optic communications link between the apparatus and the operator would eliminate a significant portion of the latency in the prior art systems. This solution also allows for greater bandwidth. However, this solution also limits the distance between the operator and the apparatus and is impractical in many situations. This solution also is vulnerable because the communication link can be broken if the optical fiber is severed. As a result, this solution is not practical in hostile operations such as military operations, operations in busy environments such as urban areas, in situations where there is other vehicle traffic which may break the optical fiber, and in situations where the apparatus may turn around and run over its own optical fiber.
Another solution proposed by the prior art is to increase the bandwidth, or the effective bandwidth, of the wireless link. While this solution can reduce the interruptions in the feedback from the apparatus, is can also create other problems such as increasing latency. For example, increasing the effective bandwidth often involves using increased data compression, which tends to increase latency by requiring additional processing of the data signals.
Other solutions involve using supervisory control where interaction with the remote apparatus is less frequent and the apparatus assumes more autonomy For example, in space exploration (such as rovers on Mars), the human involvement is infrequent because of the long transmission delays. In these situations, the remote apparatus receives input and executes those instructions. The remote apparatus then stops and waits while feedback is sent to the operator, while the operator considers the feedback, and while new instructions are sent to the apparatus. This creates frequent periods of time in which the apparatus is waiting for instructions. It raises the risk of mission failure due to reliance on the competence of the apparatus rather than the human. It also results in a slow and tedious operation that proceeds in a manner similar to that in which humans operate in high latency systems, as described above.
Accordingly, there is a need for improved methods, apparatuses, and systems for remote apparatus operation and remote sensing, particularly for methods, apparatuses, and systems in which latency is reduced or compensated and relatively low bandwidth communications links are utilized. Those and other advantages of the present invention will be described in more detail hereinbelow.